Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Visualize images from the batch #2063

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Visualize images from the batch #2063

wants to merge 1 commit into from

Conversation

mzweilin
Copy link
Contributor

📝 Description

  • This PR visualizes images that are already loaded in a batch, instead of loading them again from the file system. This change allows visualizing adversarial images that are perturbed by a Lightning Callback.

✨ Changes

Select what type of change your PR is:

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • 🔨 Refactor (non-breaking change which refactors the code base)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔒 Security update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

  • 📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
  • 📚 I have made the necessary updates to the documentation (if applicable).
  • 🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

For more information about code review checklists, see the Code Review Checklist.

@mzweilin mzweilin force-pushed the mzweilin/visualize_images_from_batch branch from c2420fc to d76a5d1 Compare May 15, 2024 23:45
@mzweilin
Denomalize and visualize the ImageNet-normalized batch.
d76a5d1 
Signed-off-by: Weilin Xu <weilin.xu@intel.com>
ashwinvaidya17
ashwinvaidya17 requested changes Jun 27, 2024
View reviewed changes
Copy link
Collaborator

@ashwinvaidya17 ashwinvaidya17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Visualizing images from the batch is a useful feature, but we might have to consider the original normalization transforms that are specific to the model or passed to the model.

src/anomalib/utils/visualization/image.py
height, width = batch["image"].shape[-2:]
image = (read_image(path=batch["image_path"][i]) * 255).astype(np.uint8)
image = cv2.resize(image, dsize=(width, height), interpolation=cv2.INTER_AREA)
image = batch["image"][i].cpu().numpy().transpose(1, 2, 0) # HWC, RGB
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if this is the best approach. I would recommend creating a new visualizer sub-classed from

anomalib/src/anomalib/utils/visualization/base.py

Line 34 in 56843d2

class BaseVisualizer(ABC):
. You can access the batch in a similar manner as

anomalib/src/anomalib/utils/visualization/image.py

Line 124 in 56843d2

outputs = kwargs.get("outputs", None)
. This new class can then safely make the strong assumption that the images are always normalized to imagenet scales. Additionally, it will not break the functionality to load images from the file-path.
We are planning on refactoring the visualization callback so that we can pass the visualizers to the engine rather than hard-coding it but for now you can add the new visualizer to the list here

anomalib/src/anomalib/engine/engine.py

Line 437 in 56843d2

visualizers=ImageVisualizer(task=self.task, normalize=self.normalization == NormalizationMethod.NONE),

Another option is to invert the normalization transforms attached to the model

anomalib/src/anomalib/models/image/padim/lightning_model.py

Line 131 in 56843d2

Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
or from the dataloader. But it might require a more complex solution.

@alexriedel1 alexriedel1 mentioned this pull request Aug 13, 2024
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ashwinvaidya17 ashwinvaidya17 ashwinvaidya17 requested changes

@samet-akcay samet-akcay Awaiting requested review from samet-akcay samet-akcay is a code owner

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mzweilin @ashwinvaidya17